Service providers can provide services to multiple subscribers in a data network and collect reporting data related to network and subscriber behavior. The reporting data collected and delivered by network devices along a data path may need to be of high granularity and visibility. Therefore, the amount of the collected reporting data may rise due to the growth of throughput of the network devices, number of subscribers, and demand for high granularity of the reporting data collected by the network devices servicing the subscribers. The reporting data may be consolidated and processed by centralized analytics and reporting systems.
Specifically, the network devices may collect the reporting data and deliver the collected reporting data to specific network devices responsible for processing the collected reporting data and presenting a view of the network and subscriber behavior from different perspectives to the service providers. However, the centralized analytics and reporting systems may need to scale out and provide more network and computational resources for receiving and processing the growing amount of reporting data.
Additionally, multiple data records in the reporting data collected by the network devices can include duplicates, thus resulting in collecting and processing redundant data.